The present invention relates to computer programs for solving statistical problems related to large amounts of data and in particular to a computer program and method implementing incremental gradient methods within a relational database management program.
Numeric analysis programs such as, MatLab and Mathematica, have been developed to assist users in performing time consuming and complex numeric computations, such as statistical analysis, on user-supplied data. Typically such numeric analysis programs provide highly optimized numeric calculations accessing relatively small data files holding the user-supplied data in simple, static data structures loaded in random access memory on the computer running the program.
For large user-supplied data sets, it is known to link a numeric analysis program to a relational database management program that may handle access to the user data stored in a database. As is generally understood in the art, relational database management programs are programs that provide a set of optimized functions for accessing and manipulating large sets of dynamic data. Relational relational database management programs typically provide standard functions for counting, summing, averaging, sorting, grouping, data that operate efficiently at high speed. Generally relational database management programs also provide mechanisms to ensure the integrity security and recoverability of the data in an environment where the data may be readily updated and changed. The relational database management program typically enforces a particular data structure or grouping of data on the physical storage media to improve data access speed and compactness.
Connecting numeric analysis programs to relational database management programs can be relatively inefficient and may require exporting of the needed data from the database into a static datafile, piece-by-piece for execution on the numeric analysis program and then a re-importing of the data back into the database. This approach may be error-prone, difficult, and slow.